Chat

With our chat/completions-endpoint you can generate completions in a conversational style. The prompted model will answer to user prompts similarly to how a human would. It is possible to have multi-turn conversations, i.e. conversations that contain follow-up questions to intermediate responses.

Code Example

Our API endpoint is compatible with OpenAI's ChatGPT endpoint in most parameters. We, therefore, do not provide an own Python client to our users but, instead, recommend to use an existing OpenAI-compatible client. The following example demonstrates the usage of the official OpenAI client against our API.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AA_TOKEN"],

    # The official SaaS URL or your own on-premise URL.
    base_url="https://api.aleph-alpha.com",
)

messages = client.chat.completions.create(
    # Pick a model that is chat-capable.
    model="llama-3.1-70b-instruct",

    # Activate streaming. This parameter is optional.
    stream=True,

    # Send the entire past conversation as well as your follow-up question.
    messages=[
        {
            "role": "user",
            "content": "What is answer to the ultimate question of life, the universe, and everything? Be brief.",
        },
        {
            "role": "assistant",
            "content": "42.",
        },
        {
            "role": "user",
            "content": "Why was that number chosen?",
        },
    ],
)

for message in messages:
    content = message.choices[0].delta.content
    if content != None:
        print(content)
    else:
        print("<end-of-stream>")

# prints:
# According to Douglas Adams, the author of "The Hitchhiker's Guide to
#  the Galaxy," the book in which the answer "42" appears, the number was
#  chosen randomly and for its comedic effect. Adams wanted a number that was ordinary and un
# remarkable, yet somehow profound and mysterious. He has said in interviews that he chose
#  42 because it was a number that didn't have any particular significance or connotation
# , and he thought it would be humorous to present it as the answer to the ultimate
#  question of life, the universe, and everything.
# <end-of-stream>

If you need more information on the parameters you can use, please checkout our HTTP API.